feat(gepa): add tool description optimization for multi-agent systems #8928

Ju-usc · 2025-10-10T06:12:23Z

Summary

Addresses #8706 which requested GEPA to optimize tool descriptions. This PR expands on that to enable comprehensive ReAct module optimization with joint optimization of all four ReAct components: react instructions, extract instructions, tool descriptions, and tool argument descriptions.

When optimize_react_components=True, GEPA discovers all dspy.ReAct modules in your program (including nested multi-agent systems) and uses a specialized reflection prompt to jointly optimize how agents reason, select tools, and extract answers from execution trajectories. All ReAct components are optimized together based on shared execution traces, enabling the reflection LM to generate cohesive instructions since it sees how components work together (not optimized in isolation). This addresses the ReAct trajectory prefix duplication issue (gepa-ai/gepa#97).

Fully backward compatible - Default optimize_react_components=False preserves existing behavior.

Issue

Closes #8706 - Original request was to enable GEPA to optimize tool descriptions. This PR expands on that to optimize all four ReAct components jointly (react instructions, extract instructions, tool descriptions, and tool argument descriptions) for more effective agent optimization.

Changes

Core Implementation

Add optimize_react_components parameter to GEPA (default False for backward compatibility)
Unified ReAct module optimization - Treats each dspy.ReAct as one module with react/extract/tools as subcomponents, respecting both GEPA's module-level abstraction and DSPy's ReAct module design
Efficient reflective dataset - Single trajectory per ReAct execution shared across all components (eliminates duplicate trajectory formatting for separate components)
ReActModuleProposer with dynamic signatures - Specialized proposer that generates output fields for each tool/parameter, enabling selective optimization
ReAct module discovery - Traverse program via named_sub_modules() to find all dspy.ReAct instances (supports deeply nested multi-agent architectures)
Component serialization - Serialize ReAct modules as JSON configs containing react/extract instructions and tool schemas
Intelligent routing - Direct ReAct components to ReActModuleProposer, regular predictors to default/custom proposers
Component updates - Apply optimized react/extract instructions, tool descriptions, and tool argument descriptions back to ReAct modules (propagates arg_desc to tool.args for prompt rendering)

Testing

8 comprehensive tests covering:
- Single ReAct module detection
- Multi-ReAct workflow discovery (mixed ReAct + non-ReAct modules)
- Nested orchestrator-worker patterns (hierarchical agents)
- Program building with optimized components
- Reflective dataset creation with trajectory feedback
All tests validate: Discovery logic, JSON serialization, component routing, program reconstruction

Documentation

GEPA_Advanced.md - Complete ReAct optimization guide:
- What gets optimized (4 components with selective optimization)
- When to use (5 common failure patterns)
- How it works (discovery → serialization → optimization → application)
- Usage examples (basic agent + multi-agent system)
- Custom proposer interface with reference implementation
overview.md - Brief introduction linking to advanced guide
Reflection prompt documentation - Explains progressive optimization philosophy

Usage Example

Basic ReAct Agent

import dspy

# Define tools
def search_web(query: str) -> str:
    return f"Search results for: {query}"

search_tool = dspy.Tool(search_web, name="search", desc="Searches")

# Create agent
agent = dspy.ReAct("question -> answer", tools=[search_tool])

# Optimize with GEPA
gepa = dspy.GEPA(
    metric=my_metric,  # Your evaluation metric
    reflection_lm=dspy.LM(model="gpt-5"),
    optimize_react_components=True,
    auto="medium"
)

optimized = gepa.compile(agent, trainset=trainset, valset=valset)

Multi-Agent System

import dspy

def search_web(query: str) -> str:
    return f"Search results for: {query}"

def calculate(expression: str) -> float:
    return eval(expression)

search_tool = dspy.Tool(search_web, name="search", desc="Searches")
calc_tool = dspy.Tool(calculate, name="calculator", desc="Computes")

class ResearchAssistant(dspy.Module):
    def __init__(self):
        super().__init__()
        self.researcher = dspy.ReAct("query -> findings", tools=[search_tool])
        
        def delegate_research(query: str) -> str:
            return self.researcher(query=query).findings
        
        research_tool = dspy.Tool(delegate_research, name="research", desc="Delegates")
        self.assistant = dspy.ReAct("question -> answer", tools=[research_tool, calc_tool])
    
    def forward(self, question):
        return self.assistant(question=question)

# Optimizes ALL ReAct modules and tools
gepa = dspy.GEPA(
    metric=my_metric,
    reflection_lm=dspy.LM(model="gpt-5"),
    optimize_react_components=True,
    auto="medium"
)

optimized = gepa.compile(ResearchAssistant(), trainset=trainset, valset=valset)

Key Features

Joint Optimization:

React instruction and tool descriptions are optimized together based on execution traces
The reflection LM sees how components work together (not optimized in isolation)
Can generate more cohesive instructions across all components

Selective Optimization:

Reflection LM returns None for components that should stay unchanged
Only components that need improvement are updated
Enables progressive refinement across GEPA iterations

Multi-Agent Support:

Automatically discovers nested ReAct modules
Optimizes parent and sub-agent modules cohesively
Handles complex delegation patterns

- Add optimize_tool_descriptions parameter (default False) to GEPA - Extract tool descriptions from all nested modules via named_sub_modules() - Apply optimized descriptions in DspyAdapter.build_program() - Enables holistic optimization of tools across main and subagent modules - Tests: 4 new tests, all 16 pass (4 new + 12 existing)

Ju-usc · 2025-10-10T06:16:07Z

Apologies for accidentally closing #8927

Thank you for the thorough review, @LakshyAAAgrawal! I'll address your feedback:

Since tools are categorically different from prompts, they should use a different reflection meta prompt. The default reflection meta prompt is shown here https://dspy.ai/api/optimizers/GEPA/GEPA_Advanced/#default-implementation, whereas I assume that the tool must use somewhat different meta prompt. Can you implement a propose_new_texts method that mimics the default_proposer shown in the link above for all prompts, but calls to a tool description specific prompt/signature for tool evolution.
Can you also add some description to the documentation, explaining that this feature is beneficial for React agents.
(This is not a requirement to merge the PR) Would it be possible to add a simple and short tutorial demonstrating the use and performance improvement via tool evolution?

I'll start working on items 1 and 2 and update the PR soon. Please let me know if you have any specific preferences for the tutorial format!

LakshyAAAgrawal · 2025-10-10T06:20:53Z

Thanks a lot! For the tutorial, I think you can follow the current GEPA tutorial format (load a dataset, show an example from the dataset, build a dspy program, evaluate the baseline program on testset, run GEPA with new optimization settings, show the optimized programs' prompts and tool descriptions, and finally evaluate the optimized program).

Hopefully we should be able to see a nice and large gain on agentic tasks with this amazing contribution by you!

- Add ToolProposer with GenerateImprovedToolDescription signature - Implement routing logic to separate tools from signatures - Tools use ToolProposer, signatures use custom or parent default - Backward compatible: preserves existing custom_instruction_proposer behavior - Add test verifying routing splits components correctly

- Define tool functions outside class for clarity - Match structure of simple ReAct example - Add clear comments explaining architecture - Make code more readable and maintainable

Ju-usc · 2025-10-10T09:40:59Z

Hi @LakshyAAAgrawal,

I've implemented the tool-specific proposer as requested! Here's what's included:

1. Tool-Specific Proposer Implementation ✅

Added GenerateImprovedToolDescriptionFromFeedback signature with a specialized reflection prompt
Implemented ToolProposer and SingleComponentToolProposer following the MultiModalInstructionProposer pattern
Routing logic in DspyAdapter that directs tools to ToolProposer and signatures to custom/default proposers
Fully backward compatible with existing custom instruction proposers

2. Documentation ✅

Added comprehensive section to GEPA_Advanced.md
Explains when to use tool optimization (ReAct agents, multi-agent systems)
Includes usage examples for both simple and nested agent architectures
Documents how to inspect optimized tool descriptions

Reflection Prompt Design:
The tool-specific prompt is intentionally open-ended to avoid prescriptive patterns that might lead to local minima. It asks the LM to identify patterns in successful/unsuccessful tool usage and extract domain-specific information, without suggesting specific heuristics.

Before I create a short tutorial (item #3), would you have any feedback on:

The reflection prompt design - is it general enough? Any improvements you'd suggest?
The implementation approach - does the routing logic make sense?
The documentation - anything unclear or missing?

Any feedback would be helpful before I invest time in the tutorial. Thank you!

Ju-usc · 2025-10-11T03:01:40Z

wait there is a bug in the implementation working on it to fix. Also test has to be fixed.

…euse Tools now copy ReAct's reflective data with tool-specific annotation instead of complex trajectory extraction. This 15-line approach reuses ReAct's existing context (thoughts, tool calls, observations) and adds focused annotation for each tool. Implementation: - Tools receive full ReAct reflective examples (same trajectory context) - Feedback prefixed: [Optimizing tool: 'X'] for focused optimization - Reflection LM sees complete multi-step execution traces per tool Benefits: - Simpler: 15 lines vs 70+ line extraction approach - Reuses code: No duplicate trajectory formatting logic - Same context: Tools see full ReAct execution traces - Clean: Removed all debug output Tests: - 4 focused tests following GEPA patterns (removed 1 redundant) - 226KB fixture with 34 LM + 6 reflection calls - All tests passing with gpt-5-nano traces Documentation: - Updated GEPA_Advanced.md with implementation details - Explains reflective dataset construction approach

LakshyAAAgrawal · 2025-10-11T05:07:54Z

docs/docs/api/optimizers/GEPA/GEPA_Advanced.md

+
+The `optimize_tool_descriptions` parameter enables GEPA to optimize tool descriptions in addition to signature instructions. This is particularly valuable for ReAct agents and other tool-using systems, where the quality of tool descriptions directly impacts the agent's ability to select appropriate tools for each task.
+
+Unlike signature instructions that guide reasoning strategies, tool descriptions serve a fundamentally different purpose: they help agents decide **which tool to use** in a given situation. GEPA recognizes this categorical difference and applies a specialized reflection prompt tailored for tool selection decisions.


which tool to use, when to use it, and how to use it. All three are captured by the description.

Let's avoid the word "fundamentally". One can imagine that all of tool descriptions can (and many times do) simply included in the system prompt itself.

Please also add a corresponding entry in GEPA Overview, that links to this file/section.

LakshyAAAgrawal · 2025-10-11T05:10:25Z

docs/docs/api/optimizers/GEPA/GEPA_Advanced.md

+
+Consider enabling `optimize_tool_descriptions=True` when:
+
+- **Building ReAct agents**: ReAct agents rely on tool descriptions to make action selection decisions


One should consider using this, when they use dspy.Tool anywhere in the DSPy program. Here are a few scenarios for using dspy.Tool:

docs/docs/api/optimizers/GEPA/GEPA_Advanced.md

LakshyAAAgrawal · 2025-10-11T05:14:40Z

docs/docs/api/optimizers/GEPA/GEPA_Advanced.md

+)
+```
+
+**Note:** Tool optimization is fully backward compatible. Existing programs without tools, or with `optimize_tool_descriptions=False`, continue to work exactly as before.


I don't think we need to inform users about backward compatibility here. It should be implicit that there should be no behaviour changes for any program not containing dspy.Tool.

LakshyAAAgrawal · 2025-10-11T05:54:35Z

dspy/teleprompt/gepa/gepa.py

            raised if a mismatch in module-level and predictor-level score is detected.
+        optimize_tool_descriptions: Whether to optimize tool descriptions for modules with tools 
+            (e.g., ReAct agents). When enabled, tool descriptions are included in the optimization 
+            process alongside signature instructions. Default is False.


Add a link to GEPA Advanced/Tool section

LakshyAAAgrawal · 2025-10-11T06:01:25Z

dspy/teleprompt/gepa/gepa_utils.py

                    )

            self.propose_new_texts = custom_propose_new_texts
+        elif self.optimize_tool_descriptions:


Edge case: What should happen when user tries to provide both a custom proposer, and enables optimize_tool_descriptions

LakshyAAAgrawal · 2025-10-11T06:13:31Z

dspy/teleprompt/gepa/gepa_utils.py

+                # Handle signature components - replicate proposer's default behavior
+                sig_texts = {}
+                if sig_components:
+                    from gepa.strategies.instruction_proposal import InstructionProposalSignature


This is a slight deviation from this PR, but would be a large enhancement (feel free to ignore):

Create 2 fields, self.instruction_proposal_signature and self.tool_proposer, which are initialized to the default InstructionProposalSignature and ToolProposerSignature.

Take an argument from dspy.GEPA that can override the default signature values.

LakshyAAAgrawal · 2025-10-11T06:17:15Z

dspy/teleprompt/gepa/gepa_utils.py

+        # Second pass: Process tools by copying ReAct data with annotation
+        react_module_name = None
+        for name in ret_d.keys():
+            if "react" in name.lower():


Is this robust? Might it be better to use isinstance or some other way?

LakshyAAAgrawal · 2025-10-11T06:25:21Z

dspy/teleprompt/gepa/instruction_proposal.py

+
+    Your task is to write a better description for this tool.
+
+    Read the examples carefully and identify patterns in when the tool was used successfully versus when it was misused or overlooked. Identify any domain-specific information about the tool's capabilities or appropriate usage that may not be available to the assistant in the future. The assistant may have developed effective patterns for tool selection - if so, ensure the tool description supports those patterns.


Tool use. Also suggest identifying any failure modes of the tool?

dspy/teleprompt/gepa/instruction_proposal.py

LakshyAAAgrawal · 2025-10-11T06:45:56Z

Dear @Ju-usc,

This is a great PR. Thanks a lot! I have tried to be overly critical and made too many nits. Feel free to ignore if you disagree with something. Let me know if you'd like me to address anything!

Regarding the meta prompt, overall I think it looks great. However, I suggest that as you build the tutorial, you may find that the reflection prompt needs tweaking, or the content exposed in reflective_dataset for the tool may be lacking or need improvement. This is going to be an empirical exercise, which will guide what works in the reflection meta prompts. ! Looking forward to the tutorial on this too!

You may already have thoughts about what you'd like to show in the tutorial, but if not, you may consider building off (https://kargarisaac.medium.com/building-and-optimizing-multi-agent-rag-systems-with-dspy-and-gepa-2b88b5838ce2) by @kargarisaac.

- Add GenerateImprovedToolDescriptionFromFeedback signature documentation - Include tool-aware metric example showing trajectory access - Document tool prefix annotation in feedback - Note component_selector applies to both signatures and tools - Fix 'fundamentally' language per reviewer feedback

- Separate Pass 1 (predictor examples) and Pass 2 (tool aggregation) - Clarify Generated Outputs includes full trajectory for ReAct - Fix feedback annotation format to [Tool 'name' from 'predictor_key'] - Add Component Identification & Proposer Routing section - Explain dual-proposer independence (custom proposer doesn't affect tool proposer) - Use consistent terminology: 'predictor' and 'signature instructions'

LakshyAAAgrawal · 2025-10-31T07:47:31Z

docs/docs/api/optimizers/GEPA/GEPA_Advanced.md

+
+#### Implementing a Custom Proposer for ReAct
+
+If you need custom logic, you must handle ReAct components yourself. ReAct components are stored as JSON strings containing all 4 parts:


Instead of saying this, we can say that you can start with the existing implementation at X.

LakshyAAAgrawal · 2025-10-31T07:48:25Z

Hey @Ju-usc, this looks great to me. Left a few comments, otherwise happy to merge!

Can you address the ruff issues as well?

…xcept)

…r_base_program for clarity

…for selective optimization

…Act component optimization Update the ReAct proposer's reflection signature to guide the LM toward more appropriate output granularity and selective optimization. Changes: - Add context that components are progressively optimized across iterations - Change 'and' to 'and/or' for abstraction/specificity (allows flexibility) - Refine field descriptions to guide output style: * 'ReAct instruction for reasoning and tool selection' (functional context) * 'Extract instruction for answer extraction' (functional context) * 'Purpose of tool' (focuses on high-level what/why, not verbose how) * 'Usage of parameter' (focuses on specific usage, not essay) The goal is to prevent overly verbose LM outputs (multi-paragraph tool/param descriptions) while preserving exploration capability. Field descriptions now provide functional context ('for reasoning', 'purpose', 'usage') that naturally guides appropriate scope without being prescriptive about format or length. This allows the reflection LM to determine the right level of detail based on what's needed to fix failures, aligned with GEPA's general meta-prompt philosophy.

Replace prescriptive 'minimize tool calls' example with educational progression that shows users how to write effective metrics without forcing specific objectives. Changes: - Show simple metric first (just correctness feedback) - Then show trajectory-based metric (accessing agent execution) - Use clear for-loop instead of list comprehension for readability - Follow DSPy docs conventions: answer_match variable, example/pred naming - Remove 'minimize tool calls' directive - let users decide their objectives - Add bullet points explaining what trajectory can reveal (tool selection, reasoning quality, efficiency) without prescribing how to use it - Rename section to 'Writing Metrics for ReAct Optimization' (more actionable) This aligns with GEPA's philosophy: provide general, extensible patterns that users can adapt to their specific needs. Detailed examples can be shown in tutorials rather than API documentation. Addresses PR review comment 5 about prescriptive objectives in documentation.

@LakshyAAAgrawal

…duleProposer Address PR review comment 6 by simplifying the custom proposer documentation. Changes: - Replace long inline implementation example with clickable GitHub link - Point to ReActModuleProposer as reference implementation - Add bulleted list of what the reference shows (parsing, dynamic signatures, etc.) - Keep essential JSON structure and interface documentation - Remove 100+ lines of redundant code example Benefits: - Less overwhelming for users (no duplicate code) - Single source of truth (reference implementation) - Clickable link to actual working code on GitHub - Users can copy/modify real implementation instead of example Addresses PR comment from @LakshyAAAgrawal about using reference instead of full implementation example.

Ju-usc · 2025-10-31T23:55:20Z

@LakshyAAAgrawal Thanks for the thorough review! Addressed all 6 comments:

"spy" naming → Renamed to setup_capture_for_base_program (82dee25)
Tool.func placeholder → Added explanatory comments (ca84b9d)
Optional fields → Changed to str | None = None for selective optimization (2eb8986)
Prevent verbose outputs → Refined reflection prompt with functional descriptions (bd4cdac)
Remove "minimize tool calls" → Rewrote metric example to be general/extensible (0ad4077)
Reference implementation → Replaced inline example with link to ReActModuleProposer (ef5563e, 1b10b65)

Let me know if you have any other thoughts to move this PR forward!

Improve the custom proposer documentation to be more user-friendly while maintaining technical accuracy. Changes: - Warmer, more inviting opening ("best way to start") - Concrete example with 'search' tool instead of generic placeholders - Plain English explanations for each component ("How the agent reasons...") - Clear separation: "What you can improve" vs "What to preserve" - Simpler code example with inline comments explaining ReAct vs regular - Concise "reference shows how to" bullets (3 key points) - More approachable tone without sacrificing precision This makes the advanced feature more accessible to users who need custom optimization logic beyond the defaults. Follows up on the previous commit addressing PR comment about custom proposer example.

Ju-usc · 2025-11-01T01:24:57Z

@LakshyAAAgrawal I have also updated the PR description to reflect all the changes. Ready for rereview!

…ation Sync documentation with actual reflection prompt after bd4cdac: - Add 'These components are progressively optimized' context - Change to 'and/or specificity' for flexibility - Update output field types to 'str | None' with default=None - Refine field descriptions ('for reasoning and tool selection', 'for answer extraction') - Add note about dynamic field descriptions ('Purpose of tool', 'Usage of parameter') This ensures docs accurately reflect the current prompt design that guides appropriate granularity without being prescriptive.

LakshyAAAgrawal · 2025-11-02T00:24:08Z

When GEPA is invoked on a DSPy program which has ReAct module, but does not have "optimize_react=True" set, print a warning saying that "For ReAct programs, consider using optimize_react=True which can optimize ..."

Add warning message when GEPA detects ReAct modules in the program but optimize_react_components=False. This helps users discover the ReAct optimization feature. Changes: - Always traverse modules to detect ReAct instances - If optimize_react_components=False, warn for each ReAct module found - Shows module path to help users identify what would be optimized - No behavioral changes when optimize_react_components=True Addresses maintainer feedback to make the feature more discoverable.

Ju-usc · 2025-11-03T03:38:37Z

@LakshyAAAgrawal accidentally hit "Close" while working on the warning feature. Reopened right away!

Implemented the warning. Ready for re-review when you have a chance!

Copilot

Pull Request Overview

This PR adds comprehensive support for optimizing ReAct modules in GEPA, addressing a critical bug where module paths were being truncated during detection and reconstruction. The implementation enables joint optimization of all ReAct components (react instructions, extract instructions, tool descriptions, and tool argument descriptions) while preserving full module paths for correct identification in nested multi-agent systems.

Key Changes

Added optimize_react_components flag to enable ReAct module optimization in GEPA
Implemented ReActModuleProposer for joint optimization of ReAct components using dynamic signature generation
Fixed path truncation bug by preserving full module paths (e.g., "multi_agent.orchestrator" instead of "multi_agent")

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
tests/teleprompt/test_gepa_react_optimization.py	Comprehensive test suite validating ReAct module detection, reconstruction, and reflective dataset creation with full path preservation
dspy/teleprompt/gepa/instruction_proposal.py	New `ReActModuleProposer` class and signature for optimizing ReAct components with dynamic tool field generation
dspy/teleprompt/gepa/gepa_utils.py	Enhanced `DspyAdapter` with ReAct-aware component routing, reflective dataset creation, and program building logic
dspy/teleprompt/gepa/gepa.py	Added ReAct module discovery, JSON serialization of module configs, and `optimize_react_components` parameter
docs/docs/api/optimizers/GEPA/overview.md	Brief overview of ReAct component optimization feature
docs/docs/api/optimizers/GEPA/GEPA_Advanced.md	Comprehensive guide on ReAct optimization including usage examples and custom proposer implementation

Comments suppressed due to low confidence (1)

dspy/teleprompt/gepa/gepa_utils.py:1

Corrected unmatched parenthesis - should be 'original)' → 'original'.

import json

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-11-03T09:11:13Z

dspy/teleprompt/gepa/instruction_proposal.py

+            tools_list = []
+            for tool_name, tool_info in current_tools_dict.items():
+                tool = dspy.Tool(
+                    func=lambda: None,  # Placeholder - Tool requires Callable, but only schema is used


Lambda function closure issue: all tools will share the same lambda instance. While this works because the function is never executed (only schema is used), consider using a named placeholder function or lambda *args, **kwargs: None for clarity that this is intentionally a no-op.

Suggested change

func=lambda: None, # Placeholder - Tool requires Callable, but only schema is used

func=lambda *args, **kwargs: None, # Placeholder - Tool requires Callable, but only schema is used

Copilot · 2025-11-03T09:11:13Z